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1.0 


INTRODUCTION 


The  goal  of  Phase  I  of  this  project  was  to  investigate  a  neural-network-based  smart  environmental 
monitor  (SEM)  for  multi-spectral  pattern  recognition.  The  Phase  I  project  was  to  establish  the 
system  design  and  to  experimentally  demonstrate  the  feasibility  of  our  approach.  The  objectives  of 
Phase  1  were  to: 

1 .  Study  existing  architectures  which  use  neural  networks  for  multi- spectral  pattern 
recognition. 

2 .  Select  an  optimal  system  architecture  for  environmental  monitoring  applications  and 
demonstrate  a  prototype  of  at  least  one  such  architecture. 

3 .  Demonstrate  the  feasibility  of  the  approach  by  training  and  testing  this  architecture 
prototype  on  a  specific  environmental  application  in  a  proof-of-concept  experiment 

The  goals  and  objectives  of  Phase  I  were  successfully  accomplished.  The  greatest  amount  of  effort 
was  directed  toward  developing  an  optimal  neural  network.  This  network  is  capable  of  acquiring 
and  processing  spectral  characteristics  obtained  from  a  compact  Raman  and  fluorescence 
spectrometer  and  can  be  used  with  any  one-dimensional  data  set.  Included  in  the  software  are  the 
algorithms  for  network  training  for  subsequent  spectral  recognition.  Details  of  the  Phase  I 
experiments  are  presented  in  this  report  A  demonstration  model  of  a  portable  smart  spectrometer 
was  constructed.  Physical  Optics  Corporation  (POC)  has  investigated  the  feasibility  of  using  its 
existing  technologies  in  optical  and  electronic  neural  networks  and  in  software  algorithms  for 
hyperspectral  image  processing.  A  design  concept  for  the  Phase  II  implementation  of  a 
holographic  optical  neural  network  for  high-speed  hyperspectral  image  processing  is  presented  in 
Section  6. 


2.0  SUMMARY  OF  PHASE  I  ACHIEVEMENTS 

In  Phase  1,  POC 

1 .  Developed  a  neural-network-based  algorithm  for  spectral  signal  processing.  The  algorithm 
is  capable  of  extracting  spectral  features  and  performing  identification  in  real  time.  The 
neural  network  is  user  friendly,  and  is  trained  by  sets  of  experimentally  acquired  data  for 
known  substances. 
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2.  Cooperated  with  the  Lawrence  Livermore  National  Laboratory  in  the  identification  of 
chemical  contaminants  from  Raman  spectra.  Preliminary  experimental  results  have  shown 
that  POC's  neural-network-based  algorithm  is  sensitive  to  spectral  features  and  robust  to 
noise,  even  when  the  input  spectrum  is  very  noisy  (signal-to-noise-ratio  (SNR)  less  than 
1:1). 

3.  Constructed  a  portable  demonstration  smart  spectrometer.  The  system  is  capable  of 
automatic  acquisition  of  spectral  signals  and  real-time  spectral  analysis  and  identification. 

4.  Conceptually  designed  a  holographic  optical  neural  network  (HONN)  architecture  for  high¬ 
speed  3-D  hyperspectral  image  processing.  The  proposed  HONN  system  will  be 
implemented  in  the  Phase  II  program. 

The  Phase  I  accomplishments  are  discussed  in  detail  below. 

3.0  NEURAL  NETWORK  ALGORITHM  FOR  SPECTRAL  SIGNAL 

PROCESSING 

Artificial  neural  networks  are  modeled  after  biological  neural  networks  Neural  networks 
consist  of  layers  of  processors  (neurons)  interconnected  as  shown  in  Figure  1.  The  data  enters  the 
system  from  the  input  neuron  layer.  Then  the  neural  network  performs  a  nonlinear  transformation, 
producing  a  result  represented  by  the  status  of  the  output  neuron  layer.  Neural  networks  can  be 
used  in  the  transformation  of  complex  spectral  signature  data  (in  the  input  neuron  layer)  into  a 
meaningful  output  (in  the  output  neuron  layer)  for  automatic  signal  identification.  The  input 
neuron  layer  is  subdivided  into  groups  of  neurons  representing  signals  originating  from  different 
input  sensors  or  feature  windows  resulting  from  pre-processing.  The  neurons  in  the  output  layer 
represent  the  likelihood  that  a  given  input  belongs  to  a  specific  output  class. 

The  processing  concept  of  the  neural  network  is  based  on  a  simple  function  that  is  executed  by  all 
of  the  neurons  in  each  layer  of  the  network.  Each  neuron  receives  input  from  the  neurons  of  the 
preceding  layer.  A  receiving  neuron  performs  a  weighted  summation  operation  on  all  the  inputs, 
compares  this  result  with  a  predefined  threshold,  then  generates  an  output  using  a  nonlinear 
function,  according  to  the  following  formula 


yj= 


N 


X  TijXj  Bj 
U=0 


(1) 
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Window  1 


Feature 
Window  2 


Feature  Feature 


Figure  1 

Schematic  of  the  neural  network  concept. 


where  xi  (i  =  0,  1, N)  is  the  input  to  the  neuron,  Tij  is  the  weight  associated  with  input 
neuron  xj,  qj  is  the  threshold  value,  and  F  is  the  non-linear  function. 

The  neural  network  can  learn  from  the  training  examples  to  associate  different  features  from  the 
input  in  order  to  reach  an  optimum  output  result.  The  adaptive  learning,  massive  interconnection, 
and  nonlinear  classification  capabilities  of  neural  networks  make  them  generally  more  robust  to 
noise  and  distortion,  and  more  sensitive  to  trained  features  for  signal  identification  and 
classification  than  are  conventional  methods.  Section  4.0  presents  several  examples  where  a  neural 
network  was  used  to  identify  spectral  features. 

In  the  Phase  1  project,  POC  developed  a  combination  which  uses  pre-processing  and  a  neural 
network  algorithm  for  spectral  signal  identification,  as  illustrated  in  Figure  2.  The  spectral  signals 
from  the  spectrometer  are  converted  to  a  digital  format  via  a  CCD  detector  and  a  frame  grabber. 
The  spectra  first  go  through  a  pre-processing  stage  in  which  smoothing,  normalization,  and 
windowed  cross-correlation  operations  are  performed.  These  signal  processing  techniques  remove 
the  thermal  noise  (via  smoothing),  eliminate  background  bias  signals  (via  normalization),  and 
enhance  features  of  interest  (via  windowed  cross-correlation). 

The  pre-processed  signals  are  then  sent  to  the  next  stage  for  neural  network  training  and 
identification.  The  input  neurons  are  grouped  to  represent  the  feature  windows  from  the  pre¬ 
processing  stage.  A  set  of  training  examples  is  used  to  train  the  neural  network  to  produce  correct 
responses  from  the  output  neurons  such  that  only  one  output  neuron  responds  high  and  the  rest 
low  for  a  certain  chemical  compound. 
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After  training,  the  neural  network  is  ready  to  perform  recognition.  When  an  unknown  spectrum  is 
input,  the  neural  network  immediately  generates  a  confidence  value  for  each  output  neuron.  The 
confidence  value  is  between  0  and  100%.  In  the  post-processing  stage,  the  user  can  define  the 
desired  thresholding  value  for  proper  recognition.  Output  values  above  the  threshold  will  be 
recognized  as  positive  identification,  whereas  output  values  below  the  threshold  will  be  considered 
negative.  For  confirmation,  a  rerun  maybe  requested  by  the  processor.  For  example,  the 
threshold  can  be  set  at  50%.  When  the  value  of  the  output  neuron  representing  Carbon 
Tetrachloride,  for  example,  is  85%,  then  the  system  will  conclude:  "Carbon  Tetrachloride  is 
detected  (with  85%  confidence)." 


Enhanced 


Input  Spectral  Confidence 

Spectaim  Features  Values 


Figure  2 

Schematic  diagram  of  POC's  neural-network-based  spectral  signal  processing. 


POC  has  developed  a  spectral  signal  processing  algorithm  that  employs  a  windowed-correlation 
algorithm  for  spectral  feature  extraction  and  a  three-layer  neural  network  for  spectral  signature 
identification.  The  schematic  diagram  of  the  spectral  signal  processing  algorithm  is  shown  in 
Figure  3.  The  windowed  correlator  can  be  expressed  as: 

b 

y(t)  =  Jx(x)h(t-x)dx  (2) 

a 

where  x(x)  is  the  input  spectrum,  h(x)  is  a  reference  function  that  contains  certain  unique  features 
in  the  window  region  [a,b],  and  y(t)  is  the  cross-correlation  output. 
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Figure  3 

POC's  spectral  signal  processing  algorithm. 


The  windowed-cross-correlation  function  enables  the  spectral  features  in  the  training  set  to 
correlate  with  the  input  unknown  spectrum.  This  pre-processing  step  can  suppress  the  noise  from 
the  spectrum,  thus  enhancing  the  desired  features  in  defined  windows.  The  correlation  output  is 
then  sent  to  a  neural  network  for  identification. 

POC  has  developed  a  three-layer  feed-forward  neural  network  for  spiectral  feature  identification. 
The  number  of  input  neurons  corresponds  to  the  number  of  data  points  after  the  windowed- 
correlation  operation,  about  200  ~  300  points  from  a  1024  point  raw  data  set  Currently,  the  neural 
network  has  200,  100,  and  5  neurons  in  the  input,  hidden,  and  output  layers,  respectively.  We 
used  backpropagation  training  to  train  the  network  with  spectra  which  have  high  SNR.  The 
network  can  identify  very  low  SNR  1)  unknown  spectra. 


4.0  IDENTIFICATION  OF  CHEMICAL  CONTAMINANTS  IN  RAMAN 

SPECTRA 

POC  has  applied  its  spectral  signal  processing  algorithms  to  the  identification  of  Raman  spectra. 
To  validate  the  potential  of  POC's  neural  network  technique  to  analyze  and  classify  spectral 
signatures,  POC  teamed  with  a  group  at  Lawrence  Livermore  National  Laboratory  (LLNL), 
directed  by  Dr.  Kevin  Kyle,  to  identify  the  presence  of  five  chlorinated  organic  compounds. 
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carbon  tetrachloride  (CCI4),  chloroform  (CHCI3).  dichloromethane  (DCM),  1,1,1-trichlorocthane 
(TCA),  and  1,1, 2-trichloroethylene  (TCE),  using  their  spectral  signatures  from  a  Raman 
spectrometer.  LLNL  provided  us  with  a  set  of  known  "clean"  spectra  (i.e.,  high  SNR  input)  from 
a  Raman  spectrometer  as  the  training  data  set  for  our  neural- network- based  spectral  identification 
technique.  Spectral  signatures  for  each  organic  compound  were  used  as  the  training  data  sets. 
Figure  4(a)  shows  a  clean  spectrum  of  a  mixture  of  the  five  compounds. 

LLNL  aLso  provided  us  with  a  large  number  of  spectra  of  mixtures  of  compounds  taken  in  noisy 
environments.  Five  sets  of  spectra  had  SNR  >3:1,  10  had  SNR  ~  2:1,  20  had  SNR  ~  1.5:1,  and 
20  had  SNR  <1:1.  The  first  three  sets  of  spectra  were  used  to  train  a  three  layer  neural  network, 
and  the  last  set  (SNR  <  1)  was  used  to  test  the  performance  of  the  spectral  identification  simulator. 

A  multi-channel  correlator  was  used  to  pre-process  the  input  spectra.  A  correlator  is  an  optimum 
filter  that  can  separate  the  majority  of  noise  from  the  desired  signals.  The  signals  are  the 
characteristic  peaks  in  the  Raman  spectrum  that  indicate  the  presence  of  certain  chemical  structures. 
In  a  Raman  spectrum,  two  or  three  peaks  can  often  be  selected  to  represent  one  chemical 
compound.  The  rest  of  the  spectrum  contains  no  useful  information  about  that  chemical.  We  can 
drastically  reduce  the  computational  requirement  for  correlation  operations  by  selecting  only  the 
proper  correlation  windows  in  the  spectrum  to  cover  the  characteristic  peaks. 

In  our  particular  experiment,  thirteen  windows  were  chosen  --  three  windows  each  for  CCI4, 
CHCI3,  TCE,  and  TCA,  and  one  window  for  CH2CI2. 

Our  task  was  to  classify  these  unknown,  noisy  spectra.  The  spectra  were  of  pure  individual 
organic  compounds,  a  random  mixture  of  the  four  or  five  compounds,  or  pure  noise.  Figure  4(b) 
shows  one  such  noisy  spectra.  The  objective  of  this  classification  exercise  was  to  verify  the 
robustness  and  adaptability  of  our  neural  network  technique.  POC  did  not  know  in  advance  the 
identity  of  the  noisy  input  spectra.  We  achieved  an  overall  successful  classification  rate  >90%. 
The  results  are  summarized  in  Table  1.  Notice  that  if  three  consecutive  frames  of  signals  are  co¬ 
added,  the  identification  rate  for  SNR  <  1:1  is  almost  100%! 
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Figure  4 

Examples  of  the  identification  of  Raman  spectra: 

(a)  "clean"  spectrum  of  a  mixture  of  five  chemical  compounds,  and 
(b)  rapid  identification  of  the  presence  of  five  compounds  m  a  noisy  environment  (SNR  =  1:1). 
(c)  rapid  Identification  of  the  presence  of  CCL4  and  CH2CI2  with  SNR  <11 
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Table  1 


Experimental  Results  of  Sample  Identification  by  POC's  Neural-Net- 
Based  Smart  Spectrometer 


Sample 

CCI4 

CHCI3 

CHCI2 

TCE 

TCA 

4-Mix 

5-Mix 

Input  SNR 

Correct 

ID/NR 

Correct 

ID/NR 

Correct 

ID/NR 

Correct 

ID/NR 

Correct 

ID/NR 

Correct  ID 

Correct  ID 

SNR  >  5:1 

100% 

100% 

SNR  =  2:1 

Data  Used  for  Training 

1 00% 

1 00% 

SNR  =  1:1 

1 00% 

99% 

SNR  <  1:1 

90% 

95% 

100% 

83% 

100% 

100% 

100% 

95% 

1 00% 
95% 

77% 

78% 

SNR  <  1:1* 
(Three-Frame) 

100% 

100% 

— 

— 

— 

— 

93% 

*  Three  frames  of  input  patterns  are  added  before  presentation  to  the  identification  neural 
network  algorithm. 


CCI4: 

Carbon  Tetrachloride 

5-Mix: 

Mixture  of  all  of  above 

CHCI3  : 

Chloroform 

4-Mix : 

Mixture  of  all  of  above  except  TCE 

CH2CI2  : 

Dichloromethane 

ID: 

Identification 

TCE: 

TCA: 

Trichloroethylene 

1,1. 1-Trichloroethane 

NR: 

Noise  Rejection 

Source  of  Data:  LLNL  Raman  Spectral  Data  Set  (Couriesy  of  Kevin  Kyle  and  Michael  Angel) 


5.0  DEVELOPMENT  OF  A  DEMONSTRATION  PORTABLE  SMART 

SPECTROMETER 

POC  has  completed  the  development  of  a  preliminary  demonstration  model  of  a  neural-network- 
based  smart  spectrometer. 

A  schematic  diagram  of  the  smart  spectrometer  is  shown  in  Figure  5.  The  system  consists  of  a 
laser  diode  (wavelength  =  670  nm,  power  =  10  mW)  as  the  excitation  light  source,  a  fiber  probe  to 
deliver  the  laser  light  to  the  sample  and  to  collect  the  return  light  from  the  sample  and  carry  it  to  the 
spectrometer  (Instruments  SA,  Model  CP140, 140  mm  focal  length  spectrograph).  A  CCD  camera 
(a  Sony  video  camera)  is  used  to  convert  the  light  intensity  of  the  spectrum  into  an  electrical  signal, 
and  a  frame  grabber  (Matrox,  Model  IP-8)  converts  the  analog  signal  into  digital  form.  A 
notebook  computer  runs  software  routines  that  control  and  synchronize  the  system  operations. 
The  neural  network  algorithm  performs  real-time  spectral  signature  extraction  and  identification. 
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The  frame  grabber  and  neural  network  processor  boards  can  be  inserted  in  a  docking  station 
interfaced  to  the  notebook  computer. 


Figure  5 

Schematic  diagram  of  POC's  smart  spectrometer. 


POC  has  designed  a  fiber  probe  specifically  for  the  smart  spectrometer,  as  shown  in  Figure  6.  A 
single  fiber  (400  fxm  diameter)  delivers  the  laser  excitation  light  to  the  sample.  Then  a  bundle  of 
small  fibers  (100  ^im  diameter)  collects  the  return  light  from  the  sample  and  carries  it  to  the 
spectrometer.  The  spectrometer  end  of  the  fiber  bundle  is  shaped  in  a  line  for  maximum 
throughput. 


From  Laser 

Fiber  Diode 


Figure  6 

Structure  of  the  fiber  probe. 
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The  smart  spectrometer  is  contained  in  a  briefcase,  as  shown  in  Figure  7.  A  notebook  computer 
with  a  docking  station  (containing  the  frame  grabber)  sits  on  top  of  the  briefcase.  The  computer 
can  be  put  into  the  briefcase  to  be  carried  easily.  The  size  of  the  system  is  16  x  18x7  cubic 
inches.  The  weight  of  the  system  is  about  20  lbs. 

Figure  8  shows  the  system  in  action.  The  output  of  the  smart  spectrometer  is  displayed  on  a 
regular  super  VGA  monitor  so  that  we  could  photograph  it  clearly  for  this  report.  In  this 
demonstration,  we  trained  the  system  to  recognize  a  fluorescent  light,  a  Tungsten  light,  a  laser 
source,  a  green  LED,  and  a  fluorescing  liquid  sample.  Other  light  sources  are  classified  as 
Unknown.  The  neural  network  takes  less  than  three  minutes  to  learn  a  set  of  20  training  examples 
on  the  notebook  computer.  Then  the  system  is  ready  for  recognition  operations.  When  the  fiber 
probe  is  pointed  at  an  object  and  a  key  is  pressed,  the  system  instantly  displays  the  name  of  the 
object  and  a  confidence  level  between  0  to  100%.  The  operator  can  set  a  threshold  value  for  the 
minimum  and  the  maximum  confidence  levels  for  recognition.  For  example,  the  threshold  can  be 
set  as  low  as  30%  of  the  confidence  level  so  that  no  important  object  is  missed,  or  the  threshold 
can  be  set  between  70%  and  90%  of  the  confidence  level  so  that  no  false  alarms  will  occur. 

The  demo  system  uses  off-the-shelf  components,  a  simple  mechanical  design,  and  a  software 
program  on  a  notebook  computer.  The  system  can  be  designed  to  be  more  compact,  more  versatile 
and  more  powerful  by  using  parallel  digital  signal  processors  (DSPs)  and  holographic  optical 
neural  networks  (HONNs).  The  design  for  the  Phase  n  implementation  of  a  high-speed  parallel 
hyperspectral  image  analyzer  is  presented  in  the  next  section. 
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Figure  7 

POC's  portable  smart  spectrometer. 


Figure  8 

POC's  smart  spectrometer  in  action. 
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6.0 


HOLOGRAPHIC  OPTICAL  NEURAL  NETWORK  FOR 
HYPERSPECTRAL  IMAGE  PROCESSING 


So  far,  we  have  been  dealing  with  l-D  spectral  signals.  The  total  volume  of  data  is  usually  around  . 
1 ,000  data  points,  approximately  one  to  ten  kBytes.  Software  and  DSP  hardware  can  be  used  to 
perform  real-time  analysis  on  this  small  data  set  However,  in  dealing  with  hyperspectral 
images  i.e.,  3-D  data  representations,  the  volume  of  data  to  be  analyzed  is  extremely  large. 
For  example,  a  hyperspectral  image  of  1000  x  1000  pixels  has  a  wavelength  range  of  400  to 
2400  nm,  with  a  spectral  resolution  of  10  nm  and  dynamic  range  of  10^.  The  total  volume  of  data 
is  as  high  as  10^  Bytes.  A  parallel  processing  system  is  needed  for  the  real-time  processing  of 
each  hyperspectral  image.  Based  on  many  years  of  experience  in  the  applications  of  advanced 
holographic  optical  neural  network  (HONN)  technology  to  pattern  recognition  POC 
proposes  to  use  HONN  technology  for  high-speed  hyperspectral  image  processing. 

POC  is  aware  that  hyperspectral  image  processing  has  considerable  commercial  potential.  Dr.  N. 
Lewis  and  his  research  team  at  NIH  are  presently  working  on  "real-time"  imaging  by  IR  and 
Raman  radiation  of  live  tissue  samples.  These  signals  arc  wavelength  tuned  with  an  acousto-optic 
tunable  filter  (AOTF),  but  the  problem  of  rapid  and  cost  effective  data  processing  has  not  yet  been 
solved  flO'Hl. 


The  proposed  HONN  system  is  illustrated  in  Figure  9(a).  It  includes  a  spectral  imaging  device,  for 
example,  an  AOTF  for  very  fast  wavelength  selection.  A  multiple  wavelength  image  sequence  (see 
Figure  9(b))  is  monitored  by  a  detector  array,  such  as  a  CCD  camera,  and  stored  in  an  image 
buffer  memory.  An  acousto-optic  (AO)  device  is  used  as  a  l-D  input  spatial  light  modulator  to 
display  the  spectral  content  of  each  pixel  of  the  image,  as  shown  in  Figure  9(c).  A  cylindrical  lens 
expands  the  input  waveform  vertically  to  illuminate  an  array  of  holograms.  A  set  of  spectral 
features  (see  Figure  9(d))  which  represents  a  library  of  different  chemical  components  has  been 
recorded  in  each  slice  of  the  hologram  array.  When  the  library  is  small,  the  spectral  feature  set  can 
be  generated  manually.  However,  neural  network  training  provides  a  versatile  method  for 
optimum  feature  extraction  for  the  generation  of  a  large  feature  library.  If  the  input  spectrum 
matches  the  stored  feature  vector,  then  the  hologram  generates  a  strong  correlation  signal  on  the 
output  detector  array.  The  intensity  of  the  peak  tells  the  degree  of  correlation  between  the  input 
unknown  signal  and  the  stored  feature  vector.  The  location  of  the  correlation  peak  indicates  which 
species  has  been  detected.  As  a  result,  the  proposed  architecture  shown  in  Figure  9(a)  performs 
high-speed  operations  to  correlate  an  input  signal  with  a  library  of  reference  features  in  parallel  in  a 
few  microseconds.  A  detailed  discussion  of  the  HONN  system  proposed  for  development  in 
Phase  n  will  be  included  in  the  Phase  Q  proposal. 
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Figure  9 

A  HONN  system  for  parallel  spectral  signal  processing. 
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